<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml">

<head>

<meta charset="utf-8" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="pandoc" />

<meta name="viewport" content="width=device-width, initial-scale=1">

<meta name="author" content="Greg Finak gfinak@fredhutch.org" />

<meta name="date" content="2019-03-11" />

<title>Using DataPackageR</title>



<style type="text/css">code{white-space: pre;}</style>
<style type="text/css" data-origin="pandoc">
a.sourceLine { display: inline-block; line-height: 1.25; }
a.sourceLine { pointer-events: none; color: inherit; text-decoration: inherit; }
a.sourceLine:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode { white-space: pre; position: relative; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
code.sourceCode { white-space: pre-wrap; }
a.sourceLine { text-indent: -1em; padding-left: 1em; }
}
pre.numberSource a.sourceLine
  { position: relative; left: -4em; }
pre.numberSource a.sourceLine::before
  { content: attr(data-line-number);
    position: relative; left: -1em; text-align: right; vertical-align: baseline;
    border: none; pointer-events: all; display: inline-block;
    -webkit-touch-callout: none; -webkit-user-select: none;
    -khtml-user-select: none; -moz-user-select: none;
    -ms-user-select: none; user-select: none;
    padding: 0 4px; width: 4em;
    color: #aaaaaa;
  }
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa;  padding-left: 4px; }
div.sourceCode
  {  }
@media screen {
a.sourceLine::before { text-decoration: underline; }
}
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #7d9029; } /* Attribute */
code span.bn { color: #40a070; } /* BaseN */
code span.bu { } /* BuiltIn */
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4070a0; } /* Char */
code span.cn { color: #880000; } /* Constant */
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
code span.dt { color: #902000; } /* DataType */
code span.dv { color: #40a070; } /* DecVal */
code span.er { color: #ff0000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #40a070; } /* Float */
code span.fu { color: #06287e; } /* Function */
code span.im { } /* Import */
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
code span.op { color: #666666; } /* Operator */
code span.ot { color: #007020; } /* Other */
code span.pp { color: #bc7a00; } /* Preprocessor */
code span.sc { color: #4070a0; } /* SpecialChar */
code span.ss { color: #bb6688; } /* SpecialString */
code span.st { color: #4070a0; } /* String */
code span.va { color: #19177c; } /* Variable */
code span.vs { color: #4070a0; } /* VerbatimString */
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */

</style>
<script>
// apply pandoc div.sourceCode style to pre.sourceCode instead
(function() {
  var sheets = document.styleSheets;
  for (var i = 0; i < sheets.length; i++) {
    if (sheets[i].ownerNode.dataset["origin"] !== "pandoc") continue;
    try { var rules = sheets[i].cssRules; } catch (e) { continue; }
    for (var j = 0; j < rules.length; j++) {
      var rule = rules[j];
      // check if there is a div.sourceCode rule
      if (rule.type !== rule.STYLE_RULE || rule.selectorText !== "div.sourceCode") continue;
      var style = rule.style.cssText;
      // check if color or background-color is set
      if (rule.style.color === '' || rule.style.backgroundColor === '') continue;
      // replace div.sourceCode by a pre.sourceCode rule
      sheets[i].deleteRule(j);
      sheets[i].insertRule('pre.sourceCode{' + style + '}', j);
    }
  }
})();
</script>



<style type="text/css">body {
background-color: #fff;
margin: 1em auto;
max-width: 700px;
overflow: visible;
padding-left: 2em;
padding-right: 2em;
font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
font-size: 14px;
line-height: 1.35;
}
#header {
text-align: center;
}
#TOC {
clear: both;
margin: 0 0 10px 10px;
padding: 4px;
width: 400px;
border: 1px solid #CCCCCC;
border-radius: 5px;
background-color: #f6f6f6;
font-size: 13px;
line-height: 1.3;
}
#TOC .toctitle {
font-weight: bold;
font-size: 15px;
margin-left: 5px;
}
#TOC ul {
padding-left: 40px;
margin-left: -1.5em;
margin-top: 5px;
margin-bottom: 5px;
}
#TOC ul ul {
margin-left: -2em;
}
#TOC li {
line-height: 16px;
}
table {
margin: 1em auto;
border-width: 1px;
border-color: #DDDDDD;
border-style: outset;
border-collapse: collapse;
}
table th {
border-width: 2px;
padding: 5px;
border-style: inset;
}
table td {
border-width: 1px;
border-style: inset;
line-height: 18px;
padding: 5px 5px;
}
table, table th, table td {
border-left-style: none;
border-right-style: none;
}
table thead, table tr.even {
background-color: #f7f7f7;
}
p {
margin: 0.5em 0;
}
blockquote {
background-color: #f6f6f6;
padding: 0.25em 0.75em;
}
hr {
border-style: solid;
border: none;
border-top: 1px solid #777;
margin: 28px 0;
}
dl {
margin-left: 0;
}
dl dd {
margin-bottom: 13px;
margin-left: 13px;
}
dl dt {
font-weight: bold;
}
ul {
margin-top: 0;
}
ul li {
list-style: circle outside;
}
ul ul {
margin-bottom: 0;
}
pre, code {
background-color: #f7f7f7;
border-radius: 3px;
color: #333;
white-space: pre-wrap; 
}
pre {
border-radius: 3px;
margin: 5px 0px 10px 0px;
padding: 10px;
}
pre:not([class]) {
background-color: #f7f7f7;
}
code {
font-family: Consolas, Monaco, 'Courier New', monospace;
font-size: 85%;
}
p > code, li > code {
padding: 2px 0px;
}
div.figure {
text-align: center;
}
img {
background-color: #FFFFFF;
padding: 2px;
border: 1px solid #DDDDDD;
border-radius: 3px;
border: 1px solid #CCCCCC;
margin: 0 5px;
}
h1 {
margin-top: 0;
font-size: 35px;
line-height: 40px;
}
h2 {
border-bottom: 4px solid #f7f7f7;
padding-top: 10px;
padding-bottom: 2px;
font-size: 145%;
}
h3 {
border-bottom: 2px solid #f7f7f7;
padding-top: 10px;
font-size: 120%;
}
h4 {
border-bottom: 1px solid #f7f7f7;
margin-left: 8px;
font-size: 105%;
}
h5, h6 {
border-bottom: 1px solid #ccc;
font-size: 105%;
}
a {
color: #0033dd;
text-decoration: none;
}
a:hover {
color: #6666ff; }
a:visited {
color: #800080; }
a:visited:hover {
color: #BB00BB; }
a[href^="http:"] {
text-decoration: underline; }
a[href^="https:"] {
text-decoration: underline; }

code > span.kw { color: #555; font-weight: bold; } 
code > span.dt { color: #902000; } 
code > span.dv { color: #40a070; } 
code > span.bn { color: #d14; } 
code > span.fl { color: #d14; } 
code > span.ch { color: #d14; } 
code > span.st { color: #d14; } 
code > span.co { color: #888888; font-style: italic; } 
code > span.ot { color: #007020; } 
code > span.al { color: #ff0000; font-weight: bold; } 
code > span.fu { color: #900; font-weight: bold; }  code > span.er { color: #a61717; background-color: #e3d2d2; } 
</style>

</head>

<body>




<h1 class="title toc-ignore">Using DataPackageR</h1>
<h4 class="author"><em>Greg Finak <a href="mailto:gfinak@fredhutch.org" class="email">gfinak@fredhutch.org</a></em></h4>
<h4 class="date"><em>2019-03-11</em></h4>


<div id="TOC">
<ul>
<li><a href="#purpose">Purpose</a></li>
<li><a href="#set-up-a-new-data-package.">Set up a new data package.</a><ul>
<li><a href="#whats-in-the-package-skeleton-structure">What’s in the package skeleton structure?</a></li>
<li><a href="#a-few-words-about-the-yaml-config-file">A few words about the YAML config file</a></li>
<li><a href="#where-do-i-put-my-raw-datasets">Where do I put my raw datasets?</a></li>
<li><a href="#an-api-to-read-raw-data-sets-from-within-an-r-or-rmd-procesing-script.">An API to read raw data sets from within an R or Rmd procesing script.</a></li>
<li><a href="#yaml-header-metadata-for-r-files-and-rmd-files.">YAML header metadata for R files and Rmd files.</a></li>
</ul></li>
<li><a href="#build-the-data-package.">Build the data package.</a><ul>
<li><a href="#documenting-your-data-set-changes-in-news.md">Documenting your data set changes in NEWS.md</a></li>
<li><a href="#why-not-just-use-r-cmd-build">Why not just use R CMD build?</a></li>
<li><a href="#a-log-of-the-build-process">A log of the build process</a></li>
<li><a href="#a-note-about-the-package-source-directory-after-building.">A note about the package source directory after building.</a></li>
<li><a href="#update-the-autogenerated-documentation.">Update the autogenerated documentation.</a></li>
</ul></li>
<li><a href="#installing-and-using-the-new-data-package">Installing and using the new data package</a><ul>
<li><a href="#accessing-vignettes-data-sets-and-data-set-documentation.">Accessing vignettes, data sets, and data set documentation.</a></li>
<li><a href="#using-the-dataversion">Using the DataVersion</a></li>
</ul></li>
<li><a href="#migrating-old-data-packages.">Migrating old data packages.</a><ul>
<li><a href="#create-a-datapackager.yml-file">Create a datapackager.yml file</a></li>
<li><a href="#reading-data-sets-from-rmd-files">Reading data sets from Rmd files</a></li>
</ul></li>
<li><a href="#partial-builds">Partial builds</a></li>
<li><a href="#multi-script-pipelines.">Multi-script pipelines.</a><ul>
<li><a href="#file-system-artifacts">File system artifacts</a></li>
<li><a href="#passing-data-objects-to-subsequent-scripts.">Passing data objects to subsequent scripts.</a></li>
</ul></li>
<li><a href="#next-steps">Next steps</a><ul>
<li><a href="#place-your-package-under-source-control">Place your package under source control</a></li>
</ul></li>
<li><a href="#additional-details">Additional Details</a><ul>
<li><a href="#fingerprints-of-stored-data-objects">Fingerprints of stored data objects</a></li>
</ul></li>
</ul>
</div>

<div id="purpose" class="section level2">
<h2>Purpose</h2>
<p>This vignette demonstrates how to use DataPackageR to build a data package.</p>
<p>DataPackageR aims to simplify data package construction.</p>
<p>It provides mechanisms for reproducibly preprocessing and tidying raw data into into documented, versioned, and packaged analysis-ready data sets.</p>
<p>Long-running or computationally intensive data processing can be decoupled from the usual <code>R CMD build</code> process while maintinaing <a href="https://en.wikipedia.org/wiki/Data_lineage">data lineage</a>.</p>
<p>In this vignette we will subset and package the <code>mtcars</code> data set.</p>
</div>
<div id="set-up-a-new-data-package." class="section level2">
<h2>Set up a new data package.</h2>
<p>We’ll set up a new data package based on <code>mtcars</code> example in the <a href="https://github.com/RGLab/DataPackageR/blob/master/README.md">README</a>. The <code>datapackage_skeleton()</code> API is used to set up a new package. The user needs to provide:</p>
<ul>
<li>R or Rmd code files that do data processing.</li>
<li>A list of R object names created by those code files.</li>
<li>Optionally a path to a directory of raw data (will be copied into the package).</li>
<li>Optionally a list of additional code files that may be dependencies of your R scripts.</li>
</ul>
<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb1-1" data-line-number="1"><span class="kw">library</span>(DataPackageR)</a>
<a class="sourceLine" id="cb1-2" data-line-number="2"></a>
<a class="sourceLine" id="cb1-3" data-line-number="3"><span class="co"># Let's reproducibly package up</span></a>
<a class="sourceLine" id="cb1-4" data-line-number="4"><span class="co"># the cars in the mtcars dataset</span></a>
<a class="sourceLine" id="cb1-5" data-line-number="5"><span class="co"># with speed &gt; 20.</span></a>
<a class="sourceLine" id="cb1-6" data-line-number="6"><span class="co"># Our dataset will be called cars_over_20.</span></a>
<a class="sourceLine" id="cb1-7" data-line-number="7"></a>
<a class="sourceLine" id="cb1-8" data-line-number="8"><span class="co"># Get the code file that turns the raw data</span></a>
<a class="sourceLine" id="cb1-9" data-line-number="9"><span class="co"># to our packaged and processed analysis-ready dataset.</span></a>
<a class="sourceLine" id="cb1-10" data-line-number="10">processing_code &lt;-</a>
<a class="sourceLine" id="cb1-11" data-line-number="11"><span class="st">  </span><span class="kw">system.file</span>(<span class="st">&quot;extdata&quot;</span>, </a>
<a class="sourceLine" id="cb1-12" data-line-number="12">              <span class="st">&quot;tests&quot;</span>,</a>
<a class="sourceLine" id="cb1-13" data-line-number="13">              <span class="st">&quot;subsetCars.Rmd&quot;</span>,</a>
<a class="sourceLine" id="cb1-14" data-line-number="14">              <span class="dt">package =</span> <span class="st">&quot;DataPackageR&quot;</span>)</a>
<a class="sourceLine" id="cb1-15" data-line-number="15"></a>
<a class="sourceLine" id="cb1-16" data-line-number="16"><span class="co"># Create the package framework.</span></a>
<a class="sourceLine" id="cb1-17" data-line-number="17">DataPackageR<span class="op">::</span><span class="kw">datapackage_skeleton</span>(<span class="dt">name =</span> <span class="st">&quot;mtcars20&quot;</span>,</a>
<a class="sourceLine" id="cb1-18" data-line-number="18">  <span class="dt">force =</span> <span class="ot">TRUE</span>,</a>
<a class="sourceLine" id="cb1-19" data-line-number="19">  <span class="dt">code_files =</span> processing_code,</a>
<a class="sourceLine" id="cb1-20" data-line-number="20">  <span class="dt">r_object_names =</span> <span class="st">&quot;cars_over_20&quot;</span>,</a>
<a class="sourceLine" id="cb1-21" data-line-number="21">  <span class="dt">path =</span> <span class="kw">tempdir</span>() </a>
<a class="sourceLine" id="cb1-22" data-line-number="22">  <span class="co">#dependencies argument is empty</span></a>
<a class="sourceLine" id="cb1-23" data-line-number="23">  <span class="co">#raw_data_dir argument is empty.</span></a>
<a class="sourceLine" id="cb1-24" data-line-number="24">  ) </a></code></pre></div>
<div id="whats-in-the-package-skeleton-structure" class="section level3">
<h3>What’s in the package skeleton structure?</h3>
<p>This has created a datapackage source tree named “mtcars20” (in a temporary directory). For a real use case you would pick a <code>path</code> on your filesystem where you could then initialize a new github repository for the package.</p>
<p>The contents of <code>mtcars20</code> are:</p>
<pre><code>Registered S3 methods overwritten by 'ggplot2':
  method         from 
  [.quosures     rlang
  c.quosures     rlang
  print.quosures rlang
                levelName
1  mtcars20              
2   ¦--DESCRIPTION       
3   ¦--R                 
4   ¦--Read-and-delete-me
5   ¦--data              
6   ¦--data-raw          
7   ¦   °--subsetCars.Rmd
8   ¦--datapackager.yml  
9   ¦--inst              
10  ¦   °--extdata       
11  °--man               </code></pre>
<p>You should fill out the <code>DESCRIPTION</code> file to describe your data package. It contains a new <code>DataVersion</code> string that will be automatically incremented when the data package is built <em>if the packaged data has changed</em>.</p>
<p>The user-provided code files reside in <code>data-raw</code>. They are executed during the data package build process.</p>
</div>
<div id="a-few-words-about-the-yaml-config-file" class="section level3">
<h3>A few words about the YAML config file</h3>
<p>A <code>datapackager.yml</code> file is used to configure and control the build process.</p>
<p>The contents are:</p>
<pre><code>configuration:
  files:
    subsetCars.Rmd:
      enabled: yes
  objects: cars_over_20
  render_root:
    tmp: '69649'</code></pre>
<p>The two main pieces of information in the configuration are a list of the files to be processed and the data sets the package will store.</p>
<p>This example packages an R data set named <code>cars_over_20</code> (the name was passed in to <code>datapackage_skeleton()</code>). It is created by the <code>subsetCars.Rmd</code> file.</p>
<p>The objects must be listed in the yaml configuration file. <code>datapackage_skeleton()</code> ensures this is done for you automatically.</p>
<p>DataPackageR provides an API for modifying this file, so it does not need to be done by hand.</p>
<p>Further information on the contents of the YAML configuration file, and the API are in the <a href="YAML_CONFIG.md">YAML Configuration Details</a></p>
</div>
<div id="where-do-i-put-my-raw-datasets" class="section level3">
<h3>Where do I put my raw datasets?</h3>
<p>Raw data (provided the size is not prohibitive) can be placed in <code>inst/extdata</code>.</p>
<p>The <code>datapackage_skeleton()</code> API has the <code>raw_data_dir</code> argument, which will copy the contents of <code>raw_data_dir</code> (and its subdirectories) into <code>inst/extdata</code> automatically.</p>
<p>In this example we are reading the <code>mtcars</code> data set that is already in memory, rather than from the file system.</p>
</div>
<div id="an-api-to-read-raw-data-sets-from-within-an-r-or-rmd-procesing-script." class="section level3">
<h3>An API to read raw data sets from within an R or Rmd procesing script.</h3>
<p>As stated in the README, in order for your processing scripts to be portable, you should not use absolute paths to files. DataPackageR provides an API to point to the data package root directory and the <code>inst/extdata</code> and <code>data</code> subdirectories. These are useful for constructing portable paths in your code to read files from these locations.</p>
<p>For example: to construct a path to a file named “mydata.csv” located in <code>inst/extdata</code> in your data package source tree:</p>
<ul>
<li>use <code>DataPackageR::project_extdata_path(&quot;mydata.csv&quot;)</code> in your <code>R</code> or <code>Rmd</code> file. This would return: e.g., /var/folders/jh/x0h3v3pd4dd497g3gtzsm8500000gn/T//Rtmp765KFQ/mtcars20/inst/extdata/mydata.csv</li>
</ul>
<p>Similarly:</p>
<ul>
<li><code>DataPackageR::project_path()</code> constructs a path to the data package root directory. (e.g., /var/folders/jh/x0h3v3pd4dd497g3gtzsm8500000gn/T//Rtmp765KFQ/mtcars20)</li>
<li><code>DataPackageR::project_data_path()</code> constructs a path to the data package <code>data</code> subdirectory. (e.g., /var/folders/jh/x0h3v3pd4dd497g3gtzsm8500000gn/T//Rtmp765KFQ/mtcars20/data)</li>
</ul>
<p>Raw data sets that are stored externally (outside the data package source tree) can be constructed relative to the <code>project_path()</code>.</p>
</div>
<div id="yaml-header-metadata-for-r-files-and-rmd-files." class="section level3">
<h3>YAML header metadata for R files and Rmd files.</h3>
<p>If your processing scripts are Rmd files, the usual yaml header for rmarkdown documents should be present.</p>
<p>If you have Rmd files, you can still include a yaml header, but it should be commented with <code>#'</code> and it should be at the top of your R file. For example, a test R file in the DataPackageR package looks as follows:</p>
<pre><code>#'---
#'title: Sample report  from R script
#'author: Greg Finak
#'date: August 1, 2018
#'---
data &lt;- runif(100)</code></pre>
<p>This will be converted to an Rmd file with a proper yaml header, which will then be turned into a vignette and indexed in the built package.</p>
</div>
</div>
<div id="build-the-data-package." class="section level2">
<h2>Build the data package.</h2>
<p>Once the skeleton framework is set up,</p>
<div class="sourceCode" id="cb5"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb5-1" data-line-number="1"><span class="co"># Run the preprocessing code to build cars_over_20</span></a>
<a class="sourceLine" id="cb5-2" data-line-number="2"><span class="co"># and reproducibly enclose it in a package.</span></a>
<a class="sourceLine" id="cb5-3" data-line-number="3">DataPackageR<span class="op">:::</span><span class="kw">package_build</span>(<span class="kw">file.path</span>(<span class="kw">tempdir</span>(),<span class="st">&quot;mtcars20&quot;</span>))</a>
<a class="sourceLine" id="cb5-4" data-line-number="4"></a>
<a class="sourceLine" id="cb5-5" data-line-number="5">✔ Setting active project to <span class="st">'/private/var/folders/jh/x0h3v3pd4dd497g3gtzsm8500000gn/T/Rtmp765KFQ/mtcars20'</span></a>
<a class="sourceLine" id="cb5-6" data-line-number="6">✔ <span class="dv">1</span> data <span class="kw">set</span>(s) created by subsetCars.Rmd</a>
<a class="sourceLine" id="cb5-7" data-line-number="7">• cars_over_<span class="dv">20</span></a>
<a class="sourceLine" id="cb5-8" data-line-number="8">☘ Built  all datasets<span class="op">!</span></a>
<a class="sourceLine" id="cb5-9" data-line-number="9">Non<span class="op">-</span>interactive NEWS.md file update.</a>
<a class="sourceLine" id="cb5-10" data-line-number="10"></a>
<a class="sourceLine" id="cb5-11" data-line-number="11">✔ Creating <span class="st">'vignettes/'</span></a>
<a class="sourceLine" id="cb5-12" data-line-number="12">✔ Creating <span class="st">'inst/doc/'</span></a>
<a class="sourceLine" id="cb5-13" data-line-number="13">First time using roxygen2. Upgrading automatically...</a>
<a class="sourceLine" id="cb5-14" data-line-number="14">Updating roxygen version <span class="cf">in</span> <span class="op">/</span>private<span class="op">/</span>var<span class="op">/</span>folders<span class="op">/</span>jh<span class="op">/</span>x0h3v3pd4dd497g3gtzsm8500000gn<span class="op">/</span>T<span class="op">/</span>Rtmp765KFQ<span class="op">/</span>mtcars20<span class="op">/</span>DESCRIPTION</a>
<a class="sourceLine" id="cb5-15" data-line-number="15">Writing NAMESPACE</a>
<a class="sourceLine" id="cb5-16" data-line-number="16">Loading mtcars20</a>
<a class="sourceLine" id="cb5-17" data-line-number="17">Writing mtcars20.Rd</a>
<a class="sourceLine" id="cb5-18" data-line-number="18">Writing cars_over_<span class="fl">20.</span>Rd</a>
<a class="sourceLine" id="cb5-19" data-line-number="19">  </a>
<a class="sourceLine" id="cb5-20" data-line-number="20">   checking <span class="cf">for</span> file ‘<span class="op">/</span>private<span class="op">/</span>var<span class="op">/</span>folders<span class="op">/</span>jh<span class="op">/</span>x0h3v3pd4dd497g3gtzsm8500000gn<span class="op">/</span>T<span class="op">/</span>Rtmp765KFQ<span class="op">/</span>mtcars20<span class="op">/</span>DESCRIPTION’ ...</a>
<a class="sourceLine" id="cb5-21" data-line-number="21">  </a>
<a class="sourceLine" id="cb5-22" data-line-number="22">✔  checking <span class="cf">for</span> file ‘<span class="op">/</span>private<span class="op">/</span>var<span class="op">/</span>folders<span class="op">/</span>jh<span class="op">/</span>x0h3v3pd4dd497g3gtzsm8500000gn<span class="op">/</span>T<span class="op">/</span>Rtmp765KFQ<span class="op">/</span>mtcars20<span class="op">/</span>DESCRIPTION’</a>
<a class="sourceLine" id="cb5-23" data-line-number="23"></a>
<a class="sourceLine" id="cb5-24" data-line-number="24">  </a>
<a class="sourceLine" id="cb5-25" data-line-number="25">─  preparing ‘mtcars20’<span class="op">:</span></a>
<a class="sourceLine" id="cb5-26" data-line-number="26"></a>
<a class="sourceLine" id="cb5-27" data-line-number="27"><span class="st">  </span></a>
<a class="sourceLine" id="cb5-28" data-line-number="28"><span class="st">   </span>checking DESCRIPTION meta<span class="op">-</span>information ...</a>
<a class="sourceLine" id="cb5-29" data-line-number="29">  </a>
<a class="sourceLine" id="cb5-30" data-line-number="30">✔  checking DESCRIPTION meta<span class="op">-</span>information</a>
<a class="sourceLine" id="cb5-31" data-line-number="31"></a>
<a class="sourceLine" id="cb5-32" data-line-number="32">  </a>
<a class="sourceLine" id="cb5-33" data-line-number="33">─  checking <span class="cf">for</span> LF line<span class="op">-</span>endings <span class="cf">in</span> source and make files and shell scripts</a>
<a class="sourceLine" id="cb5-34" data-line-number="34"></a>
<a class="sourceLine" id="cb5-35" data-line-number="35">  </a>
<a class="sourceLine" id="cb5-36" data-line-number="36">─  checking <span class="cf">for</span> empty or unneeded directories</a>
<a class="sourceLine" id="cb5-37" data-line-number="37"></a>
<a class="sourceLine" id="cb5-38" data-line-number="38">  </a>
<a class="sourceLine" id="cb5-39" data-line-number="39">─  looking to see <span class="cf">if</span> a ‘data<span class="op">/</span>datalist’ file should be added</a>
<a class="sourceLine" id="cb5-40" data-line-number="40"></a>
<a class="sourceLine" id="cb5-41" data-line-number="41">  </a>
<a class="sourceLine" id="cb5-42" data-line-number="42">     NB<span class="op">:</span><span class="st"> </span>this package now depends on <span class="kw">R</span> (<span class="op">&gt;=</span><span class="st"> </span><span class="dv">3</span>.<span class="fl">5.0</span>)</a>
<a class="sourceLine" id="cb5-43" data-line-number="43"></a>
<a class="sourceLine" id="cb5-44" data-line-number="44">  </a>
<a class="sourceLine" id="cb5-45" data-line-number="45">     WARNING<span class="op">:</span><span class="st"> </span>Added dependency on R <span class="op">&gt;=</span><span class="st"> </span><span class="dv">3</span>.<span class="fl">5.0</span> because serialized objects <span class="cf">in</span>  serialize<span class="op">/</span>load version <span class="dv">3</span> cannot be read <span class="cf">in</span> older versions of R.  <span class="kw">File</span>(s) containing such objects<span class="op">:</span><span class="st"> 'mtcars20/data/cars_over_20.rda'</span></a>
<a class="sourceLine" id="cb5-46" data-line-number="46"></a>
<a class="sourceLine" id="cb5-47" data-line-number="47">  </a>
<a class="sourceLine" id="cb5-48" data-line-number="48">─  building <span class="st">'mtcars20_1.0.tar.gz'</span></a>
<a class="sourceLine" id="cb5-49" data-line-number="49"></a>
<a class="sourceLine" id="cb5-50" data-line-number="50">  </a>
<a class="sourceLine" id="cb5-51" data-line-number="51">   </a>
<a class="sourceLine" id="cb5-52" data-line-number="52"></a>
<a class="sourceLine" id="cb5-53" data-line-number="53">Next Steps </a>
<a class="sourceLine" id="cb5-54" data-line-number="54"><span class="fl">1.</span> Update your package documentation. </a>
<a class="sourceLine" id="cb5-55" data-line-number="55">   <span class="op">-</span><span class="st"> </span>Edit the documentation.R file <span class="cf">in</span> the package source data<span class="op">-</span>raw subdirectory and update the roxygen markup. </a>
<a class="sourceLine" id="cb5-56" data-line-number="56">   <span class="op">-</span><span class="st"> </span>Rebuild the package documentation with  <span class="kw">document</span>() . </a>
<a class="sourceLine" id="cb5-57" data-line-number="57"><span class="fl">2.</span> Add your package to source control. </a>
<a class="sourceLine" id="cb5-58" data-line-number="58">   <span class="op">-</span><span class="st"> </span>Call  git init .  <span class="cf">in</span> the package source root directory. </a>
<a class="sourceLine" id="cb5-59" data-line-number="59">   <span class="op">-</span><span class="st">  </span>git add  the package files. </a>
<a class="sourceLine" id="cb5-60" data-line-number="60">   <span class="op">-</span><span class="st">  </span>git commit  your new package. </a>
<a class="sourceLine" id="cb5-61" data-line-number="61">   <span class="op">-</span><span class="st"> </span>Set up a github repository <span class="cf">for</span> your pacakge. </a>
<a class="sourceLine" id="cb5-62" data-line-number="62">   <span class="op">-</span><span class="st"> </span>Add the github repository as a remote of your local package repository. </a>
<a class="sourceLine" id="cb5-63" data-line-number="63">   <span class="op">-</span><span class="st">  </span>git push  your local repository to gitub. </a>
<a class="sourceLine" id="cb5-64" data-line-number="64">[<span class="dv">1</span>] <span class="st">&quot;/private/var/folders/jh/x0h3v3pd4dd497g3gtzsm8500000gn/T/Rtmp765KFQ/mtcars20_1.0.tar.gz&quot;</span></a></code></pre></div>
<div id="documenting-your-data-set-changes-in-news.md" class="section level3">
<h3>Documenting your data set changes in NEWS.md</h3>
<p>When you build a package in interactive mode, you will be prompted to input text describing the changes to your data package (one line).</p>
<p>These will appear in the NEWS.md file in the following format:</p>
<pre><code>DataVersion: xx.yy.zz
========
A description of your changes to the package

[The rest of the file]</code></pre>
</div>
<div id="why-not-just-use-r-cmd-build" class="section level3">
<h3>Why not just use R CMD build?</h3>
<p>If the processing script is time consuming or the data set is particularly large, then <code>R CMD build</code> would run the code each time the package is installed. In such cases, raw data may not be available, or the environment to do the data processing may not be set up for each user of the data. DataPackageR decouples data processing from package building/installation for data consumers.</p>
</div>
<div id="a-log-of-the-build-process" class="section level3">
<h3>A log of the build process</h3>
<p>DataPackageR uses the <code>futile.logger</code> package to log progress.</p>
<p>If there are errors in the processing, the script will notify you via logging to console and to <code>/private/tmp/Test/inst/extdata/Logfiles/processing.log</code>. Errors should be corrected and the build repeated.</p>
<p>If everything goes smoothly, you will have a new package built in the parent directory.</p>
<p>In this case we have a new package <code>mtcars20_1.0.tar.gz</code>.</p>
</div>
<div id="a-note-about-the-package-source-directory-after-building." class="section level3">
<h3>A note about the package source directory after building.</h3>
<p>The pacakge source directory changes after the first build.</p>
<pre><code>                         levelName
1  mtcars20                       
2   ¦--DATADIGEST                 
3   ¦--DESCRIPTION                
4   ¦--NAMESPACE                  
5   ¦--NEWS.md                    
6   ¦--R                          
7   ¦   °--mtcars20.R             
8   ¦--Read-and-delete-me         
9   ¦--data                       
10  ¦   °--cars_over_20.rda       
11  ¦--data-raw                   
12  ¦   ¦--documentation.R        
13  ¦   ¦--subsetCars.R           
14  ¦   °--subsetCars.Rmd         
15  ¦--datapackager.yml           
16  ¦--inst                       
17  ¦   ¦--doc                    
18  ¦   ¦   ¦--subsetCars.Rmd     
19  ¦   ¦   °--subsetCars.html    
20  ¦   °--extdata                
21  ¦       °--Logfiles           
22  ¦           ¦--processing.log 
23  ¦           °--subsetCars.html
24  ¦--man                        
25  ¦   ¦--cars_over_20.Rd        
26  ¦   °--mtcars20.Rd            
27  °--vignettes                  
28      °--subsetCars.Rmd         </code></pre>
</div>
<div id="update-the-autogenerated-documentation." class="section level3">
<h3>Update the autogenerated documentation.</h3>
<p>After the first build, the <code>R</code> directory contains <code>mtcars.R</code> that has autogenerated <code>roxygen2</code> markup documentation for the data package and for the packaged data <code>cars_over20</code>.</p>
<p>The processed <code>Rd</code> files can be found in <code>man</code>.</p>
<p>The autogenerated documentation source is in the <code>documentation.R</code> file in <code>data-raw</code>.</p>
<p>You should update this file to properly document your objects. Then rebuild the documentation:</p>
<div class="sourceCode" id="cb8"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb8-1" data-line-number="1"><span class="kw">document</span>(<span class="kw">file.path</span>(<span class="kw">tempdir</span>(),<span class="st">&quot;mtcars20&quot;</span>))</a>
<a class="sourceLine" id="cb8-2" data-line-number="2"></a>
<a class="sourceLine" id="cb8-3" data-line-number="3">✔ Setting active project to <span class="st">'/private/var/folders/jh/x0h3v3pd4dd497g3gtzsm8500000gn/T/Rtmp765KFQ/mtcars20'</span></a>
<a class="sourceLine" id="cb8-4" data-line-number="4">Updating mtcars20 documentation</a>
<a class="sourceLine" id="cb8-5" data-line-number="5">Loading mtcars20</a>
<a class="sourceLine" id="cb8-6" data-line-number="6">[<span class="dv">1</span>] <span class="ot">TRUE</span></a></code></pre></div>
<p>This is done without reprocessing the data.</p>
<div id="dont-forget-to-rebuild-the-package." class="section level4">
<h4>Dont’ forget to rebuild the package.</h4>
<p>You should update the documentation in <code>R/mtcars.R</code>, then call <code>package_build()</code> again.</p>
</div>
</div>
</div>
<div id="installing-and-using-the-new-data-package" class="section level2">
<h2>Installing and using the new data package</h2>
<div id="accessing-vignettes-data-sets-and-data-set-documentation." class="section level3">
<h3>Accessing vignettes, data sets, and data set documentation.</h3>
<p>The package source also contains files in the <code>vignettes</code> and <code>inst/doc</code> directories that provide a log of the data processing.</p>
<p>When the package is installed, these will be accessible via the <code>vignette()</code> API.</p>
<p>The vignette will detail the processing performed by the <code>subsetCars.Rmd</code> processing script.</p>
<p>The data set documentation will be accessible via <code>?cars_over_20</code>, and the data sets via <code>data()</code>.</p>
<div class="sourceCode" id="cb9"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb9-1" data-line-number="1"><span class="co"># create a temporary library to install into.</span></a>
<a class="sourceLine" id="cb9-2" data-line-number="2"><span class="kw">dir.create</span>(<span class="kw">file.path</span>(<span class="kw">tempdir</span>(),<span class="st">&quot;lib&quot;</span>))</a>
<a class="sourceLine" id="cb9-3" data-line-number="3"><span class="co"># Let's use the package we just created.</span></a>
<a class="sourceLine" id="cb9-4" data-line-number="4"><span class="kw">install.packages</span>(<span class="kw">file.path</span>(<span class="kw">tempdir</span>(),<span class="st">&quot;mtcars20_1.0.tar.gz&quot;</span>), <span class="dt">type =</span> <span class="st">&quot;source&quot;</span>, <span class="dt">repos =</span> <span class="ot">NULL</span>, <span class="dt">lib =</span> <span class="kw">file.path</span>(<span class="kw">tempdir</span>(),<span class="st">&quot;lib&quot;</span>))</a>
<a class="sourceLine" id="cb9-5" data-line-number="5"><span class="cf">if</span>(<span class="op">!</span><span class="st">&quot;package:mtcars20&quot;</span><span class="op">%in%</span><span class="kw">search</span>())</a>
<a class="sourceLine" id="cb9-6" data-line-number="6">  <span class="kw">attachNamespace</span>(<span class="st">'mtcars20'</span>) <span class="co">#use library() in your code</span></a>
<a class="sourceLine" id="cb9-7" data-line-number="7"><span class="kw">data</span>(<span class="st">&quot;cars_over_20&quot;</span>) <span class="co"># load the data</span></a>
<a class="sourceLine" id="cb9-8" data-line-number="8"></a>
<a class="sourceLine" id="cb9-9" data-line-number="9">cars_over_<span class="dv">20</span> <span class="co"># now we can use it.</span></a>
<a class="sourceLine" id="cb9-10" data-line-number="10">   speed dist</a>
<a class="sourceLine" id="cb9-11" data-line-number="11"><span class="dv">44</span>    <span class="dv">22</span>   <span class="dv">66</span></a>
<a class="sourceLine" id="cb9-12" data-line-number="12"><span class="dv">45</span>    <span class="dv">23</span>   <span class="dv">54</span></a>
<a class="sourceLine" id="cb9-13" data-line-number="13"><span class="dv">46</span>    <span class="dv">24</span>   <span class="dv">70</span></a>
<a class="sourceLine" id="cb9-14" data-line-number="14"><span class="dv">47</span>    <span class="dv">24</span>   <span class="dv">92</span></a>
<a class="sourceLine" id="cb9-15" data-line-number="15"><span class="dv">48</span>    <span class="dv">24</span>   <span class="dv">93</span></a>
<a class="sourceLine" id="cb9-16" data-line-number="16"><span class="dv">49</span>    <span class="dv">24</span>  <span class="dv">120</span></a>
<a class="sourceLine" id="cb9-17" data-line-number="17"><span class="dv">50</span>    <span class="dv">25</span>   <span class="dv">85</span></a>
<a class="sourceLine" id="cb9-18" data-line-number="18">?cars_over_<span class="dv">20</span> <span class="co"># See the documentation you wrote in data-raw/documentation.R.</span></a>
<a class="sourceLine" id="cb9-19" data-line-number="19">  </a>
<a class="sourceLine" id="cb9-20" data-line-number="20">vignettes &lt;-<span class="st"> </span><span class="kw">vignette</span>(<span class="dt">package =</span> <span class="st">&quot;mtcars20&quot;</span>)</a>
<a class="sourceLine" id="cb9-21" data-line-number="21">vignettes<span class="op">$</span>results</a>
<a class="sourceLine" id="cb9-22" data-line-number="22">      Package   </a>
<a class="sourceLine" id="cb9-23" data-line-number="23">Topic <span class="st">&quot;mtcars20&quot;</span></a>
<a class="sourceLine" id="cb9-24" data-line-number="24">      LibPath                                                         </a>
<a class="sourceLine" id="cb9-25" data-line-number="25">Topic <span class="st">&quot;/Library/Frameworks/R.framework/Versions/3.6/Resources/library&quot;</span></a>
<a class="sourceLine" id="cb9-26" data-line-number="26">      Item         Title                                            </a>
<a class="sourceLine" id="cb9-27" data-line-number="27">Topic <span class="st">&quot;subsetCars&quot;</span> <span class="st">&quot;A Test Document for DataPackageR (source, html)&quot;</span></a></code></pre></div>
</div>
<div id="using-the-dataversion" class="section level3">
<h3>Using the DataVersion</h3>
<p>Your downstream data analysis can depend on a specific version of the data in your data package by testing the DataVersion string in the DESCRIPTION file.</p>
<p>We provide an API for this:</p>
<div class="sourceCode" id="cb10"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb10-1" data-line-number="1"><span class="co"># We can easily check the version of the data</span></a>
<a class="sourceLine" id="cb10-2" data-line-number="2">DataPackageR<span class="op">::</span><span class="kw">data_version</span>(<span class="st">&quot;mtcars20&quot;</span>)</a>
<a class="sourceLine" id="cb10-3" data-line-number="3">[<span class="dv">1</span>] <span class="st">'0.1.0'</span></a>
<a class="sourceLine" id="cb10-4" data-line-number="4"></a>
<a class="sourceLine" id="cb10-5" data-line-number="5"><span class="co"># You can use an assert to check the data version in  reports and</span></a>
<a class="sourceLine" id="cb10-6" data-line-number="6"><span class="co"># analyses that use the packaged data.</span></a>
<a class="sourceLine" id="cb10-7" data-line-number="7"><span class="kw">assert_data_version</span>(<span class="dt">data_package_name =</span> <span class="st">&quot;mtcars20&quot;</span>,</a>
<a class="sourceLine" id="cb10-8" data-line-number="8">                    <span class="dt">version_string =</span> <span class="st">&quot;0.1.0&quot;</span>,</a>
<a class="sourceLine" id="cb10-9" data-line-number="9">                    <span class="dt">acceptable =</span> <span class="st">&quot;equal&quot;</span>)  <span class="co">#If this fails, execution stops</span></a>
<a class="sourceLine" id="cb10-10" data-line-number="10">                                           <span class="co">#and provides an informative error.</span></a></code></pre></div>
</div>
</div>
<div id="migrating-old-data-packages." class="section level1">
<h1>Migrating old data packages.</h1>
<p>Version 1.12.0 has moved away from controlling the build process using <code>datasets.R</code> and an additional <code>masterfile</code> argument.</p>
<p>The build process is now controlled via a <code>datapackager.yml</code> configuration file located in the package root directory. (see <a href="YAML_CONFIG.md">YAML Configuration Details</a>)</p>
<div id="create-a-datapackager.yml-file" class="section level3">
<h3>Create a datapackager.yml file</h3>
<p>You can migrate an old package by constructing such a config file using the <code>construct_yml_config()</code> API.</p>
<div class="sourceCode" id="cb11"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb11-1" data-line-number="1"><span class="co"># assume I have file1.Rmd and file2.R located in /data-raw, </span></a>
<a class="sourceLine" id="cb11-2" data-line-number="2"><span class="co"># and these create 'object1' and 'object2' respectively.</span></a>
<a class="sourceLine" id="cb11-3" data-line-number="3"></a>
<a class="sourceLine" id="cb11-4" data-line-number="4">config &lt;-<span class="st"> </span><span class="kw">construct_yml_config</span>(<span class="dt">code =</span> <span class="kw">c</span>(<span class="st">&quot;file1.Rmd&quot;</span>, <span class="st">&quot;file2.R&quot;</span>),</a>
<a class="sourceLine" id="cb11-5" data-line-number="5">                              <span class="dt">data =</span> <span class="kw">c</span>(<span class="st">&quot;object1&quot;</span>, <span class="st">&quot;object2&quot;</span>))</a>
<a class="sourceLine" id="cb11-6" data-line-number="6"><span class="kw">cat</span>(yaml<span class="op">::</span><span class="kw">as.yaml</span>(config))</a>
<a class="sourceLine" id="cb11-7" data-line-number="7">configuration<span class="op">:</span></a>
<a class="sourceLine" id="cb11-8" data-line-number="8"><span class="st">  </span>files<span class="op">:</span></a>
<a class="sourceLine" id="cb11-9" data-line-number="9"><span class="st">    </span>file1.Rmd<span class="op">:</span></a>
<a class="sourceLine" id="cb11-10" data-line-number="10"><span class="st">      </span>enabled<span class="op">:</span><span class="st"> </span>yes</a>
<a class="sourceLine" id="cb11-11" data-line-number="11">    file2.R<span class="op">:</span></a>
<a class="sourceLine" id="cb11-12" data-line-number="12"><span class="st">      </span>enabled<span class="op">:</span><span class="st"> </span>yes</a>
<a class="sourceLine" id="cb11-13" data-line-number="13">  objects<span class="op">:</span></a>
<a class="sourceLine" id="cb11-14" data-line-number="14"><span class="st">  </span><span class="op">-</span><span class="st"> </span>object1</a>
<a class="sourceLine" id="cb11-15" data-line-number="15">  <span class="op">-</span><span class="st"> </span>object2</a>
<a class="sourceLine" id="cb11-16" data-line-number="16">  render_root<span class="op">:</span></a>
<a class="sourceLine" id="cb11-17" data-line-number="17"><span class="st">    </span>tmp<span class="op">:</span><span class="st"> '251712'</span></a></code></pre></div>
<p><code>config</code> is a newly constructed yaml configuration object. It can be written to the package directory:</p>
<div class="sourceCode" id="cb12"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb12-1" data-line-number="1">path_to_package &lt;-<span class="st"> </span><span class="kw">tempdir</span>() <span class="co">#e.g., if tempdir() was the root of our package.</span></a>
<a class="sourceLine" id="cb12-2" data-line-number="2"><span class="kw">yml_write</span>(config, <span class="dt">path =</span> path_to_package)</a></code></pre></div>
<p>Now the package at <code>path_to_package</code> will build with version 1.12.0 or greater.</p>
</div>
<div id="reading-data-sets-from-rmd-files" class="section level3">
<h3>Reading data sets from Rmd files</h3>
<p>In versions prior to 1.12.1 we would read data sets from <code>inst/extdata</code> in an <code>Rmd</code> script using paths relative to <code>data-raw</code> in the data package source tree.</p>
<p>For example:</p>
<div id="the-old-way" class="section level4">
<h4>The old way</h4>
<div class="sourceCode" id="cb13"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb13-1" data-line-number="1"><span class="co"># read 'myfile.csv' from inst/extdata relative to data-raw where the Rmd is rendered.</span></a>
<a class="sourceLine" id="cb13-2" data-line-number="2"><span class="kw">read.csv</span>(<span class="kw">file.path</span>(<span class="st">&quot;../inst/extdata&quot;</span>,<span class="st">&quot;myfile.csv&quot;</span>))</a></code></pre></div>
<p>Now <code>Rmd</code> and <code>R</code> scripts are processed in <code>render_root</code> defined in the yaml config.</p>
<p>To read a raw data set we can get the path to the package source directory using an API call:</p>
</div>
<div id="the-new-way" class="section level4">
<h4>The new way</h4>
<div class="sourceCode" id="cb14"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb14-1" data-line-number="1"><span class="co"># DataPackageR::project_extdata_path() returns the path to the data package inst/extdata subdirectory directory.</span></a>
<a class="sourceLine" id="cb14-2" data-line-number="2"><span class="co"># DataPackageR::project_path() returns the path to the data package root directory.</span></a>
<a class="sourceLine" id="cb14-3" data-line-number="3"><span class="co"># DataPackageR::project_data_path() returns the path to the data package data subdirectory directory.</span></a>
<a class="sourceLine" id="cb14-4" data-line-number="4"><span class="kw">read.csv</span>(</a>
<a class="sourceLine" id="cb14-5" data-line-number="5">    DataPackageR<span class="op">::</span><span class="kw">project_extdata_path</span>(<span class="st">&quot;myfile.csv&quot;</span>)</a>
<a class="sourceLine" id="cb14-6" data-line-number="6">    )</a></code></pre></div>
</div>
</div>
</div>
<div id="partial-builds" class="section level1">
<h1>Partial builds</h1>
<p>We can also perform partial builds of a subset of files in a package by toggling the <code>enabled</code> key in the config file.</p>
<p>This can be done with the following API:</p>
<div class="sourceCode" id="cb15"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb15-1" data-line-number="1">config &lt;-<span class="st"> </span><span class="kw">yml_disable_compile</span>(config,<span class="dt">filenames =</span> <span class="st">&quot;file2.R&quot;</span>)</a>
<a class="sourceLine" id="cb15-2" data-line-number="2"><span class="kw">yml_write</span>(config, <span class="dt">path =</span> path_to_package) <span class="co"># write modified yml to the package.</span></a>
<a class="sourceLine" id="cb15-3" data-line-number="3">configuration<span class="op">:</span></a>
<a class="sourceLine" id="cb15-4" data-line-number="4"><span class="st">  </span>files<span class="op">:</span></a>
<a class="sourceLine" id="cb15-5" data-line-number="5"><span class="st">    </span>file1.Rmd<span class="op">:</span></a>
<a class="sourceLine" id="cb15-6" data-line-number="6"><span class="st">      </span>enabled<span class="op">:</span><span class="st"> </span>yes</a>
<a class="sourceLine" id="cb15-7" data-line-number="7">    file2.R<span class="op">:</span></a>
<a class="sourceLine" id="cb15-8" data-line-number="8"><span class="st">      </span>enabled<span class="op">:</span><span class="st"> </span>no</a>
<a class="sourceLine" id="cb15-9" data-line-number="9">  objects<span class="op">:</span></a>
<a class="sourceLine" id="cb15-10" data-line-number="10"><span class="st">  </span><span class="op">-</span><span class="st"> </span>object1</a>
<a class="sourceLine" id="cb15-11" data-line-number="11">  <span class="op">-</span><span class="st"> </span>object2</a>
<a class="sourceLine" id="cb15-12" data-line-number="12">  render_root<span class="op">:</span></a>
<a class="sourceLine" id="cb15-13" data-line-number="13"><span class="st">    </span>tmp<span class="op">:</span><span class="st"> '251712'</span></a></code></pre></div>
<p>Note that the modified configuration needs to be written back to the package source directory in order for the changes to take effect.</p>
<p>The consequence of toggling a file to <code>enable: no</code> is that it will be skipped when the package is rebuilt, but the data will still be retained in the package, and the documentation will not be altered.</p>
<p>This is useful in situations where we have multiple data sets, and want to re-run one script to update a specific data set, but not the other scripts because they may be too time consuming, for example.</p>
</div>
<div id="multi-script-pipelines." class="section level1">
<h1>Multi-script pipelines.</h1>
<p>We may have situations where we have mutli-script pipelines. There are two ways to share data among scripts.</p>
<ol style="list-style-type: decimal">
<li>filesystem artifacts</li>
<li>data objects passed to subsequent scripts.</li>
</ol>
<div id="file-system-artifacts" class="section level3">
<h3>File system artifacts</h3>
<p>The yaml configuration property <code>render_root</code> specifies the working directory where scripts will be rendered.</p>
<p>If a script writes files to the working directory, that is where files will appear. These can be read by subsequent scripts.</p>
</div>
<div id="passing-data-objects-to-subsequent-scripts." class="section level3">
<h3>Passing data objects to subsequent scripts.</h3>
<p>A script (e.g., <code>script2.Rmd</code>) running after <code>script1.Rmd</code> can access a stored data object named <code>script1_dataset</code> created by <code>script1.Rmd</code> by calling</p>
<p><code>script1_dataset &lt;- DataPackageR::datapackager_object_read(&quot;script1_dataset&quot;)</code>.</p>
<p>Passing of data objects amongst scripts can be turned off via:</p>
<p><code>package_build(deps = FALSE)</code></p>
</div>
</div>
<div id="next-steps" class="section level1">
<h1>Next steps</h1>
<p>We recommend the following once your package is created.</p>
<div id="place-your-package-under-source-control" class="section level2">
<h2>Place your package under source control</h2>
<p>You now have a data package source tree.</p>
<ul>
<li><strong>Place your package under version control</strong>
<ol style="list-style-type: decimal">
<li>Call <code>git init</code> in the package source root to initialize a new git repository.</li>
<li><a href="https://help.github.com/articles/create-a-repo/">Create a new repository for your data package on github</a>.</li>
<li>Push your local package repository to <code>github</code>. <a href="https://help.github.com/articles/adding-an-existing-project-to-github-using-the-command-line/">see step 7</a></li>
</ol></li>
</ul>
<p>This will let you version control your data processing code, and provide a mechanism for sharing your package with others.</p>
<p>For more details on using git and github with R, there is an excellent guide provided by Jenny Bryan: <a href="http://happygitwithr.com/">Happy Git and GitHub for the useR</a> and Hadley Wickham’s <a href="http://r-pkgs.had.co.nz/">book on R packages</a>.</p>
</div>
</div>
<div id="additional-details" class="section level1">
<h1>Additional Details</h1>
<p>We provide some additional details for the interested.</p>
<div id="fingerprints-of-stored-data-objects" class="section level3">
<h3>Fingerprints of stored data objects</h3>
<p>DataPackageR calculates an md5 checksum of each data object it stores, and keeps track of them in a file called <code>DATADIGEST</code>.</p>
<ul>
<li>Each time the package is rebuilt, the md5 sums of the new data objects are compared against the DATADIGEST.</li>
<li>If they don’t match, the build process checks that the <code>DataVersion</code> string has been incremented in the <code>DESCRIPTION</code> file.</li>
<li>If it has not the build process will exit and produce an error message.</li>
</ul>
<div id="datadigest" class="section level4">
<h4>DATADIGEST</h4>
<p>The <code>DATADIGEST</code> file contains the following:</p>
<pre><code>DataVersion: 0.1.0
cars_over_20: 3ccb5b0aaa74fe7cfc0d3ca6ab0b5cf3</code></pre>
</div>
<div id="description" class="section level4">
<h4>DESCRIPTION</h4>
<p>The description file has the new <code>DataVersion</code> string.</p>
<pre><code>Package: mtcars20
Title: What the Package Does (One Line, Title Case)
Version: 1.0
Authors@R: 
    person(given = &quot;First&quot;,
           family = &quot;Last&quot;,
           role = c(&quot;aut&quot;, &quot;cre&quot;),
           email = &quot;first.last@example.com&quot;)
Description: What the package does (one paragraph).
License: What license it uses
Encoding: UTF-8
LazyData: true
DataVersion: 0.1.0
Roxygen: list(markdown = TRUE)
Date: 2019-03-11
Suggests: 
    knitr,
    rmarkdown
VignetteBuilder: knitr
RoxygenNote: 6.1.1</code></pre>
</div>
</div>
</div>



<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
  (function () {
    var script = document.createElement("script");
    script.type = "text/javascript";
    script.src  = "https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML";
    document.getElementsByTagName("head")[0].appendChild(script);
  })();
</script>

</body>
</html>
